Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: abstract around base package manager #22

Merged
merged 1 commit into from
Sep 10, 2024

Conversation

jedel1043
Copy link
Contributor

@jedel1043 jedel1043 commented Aug 19, 2024

This PR abstracts around the base package manager used.

The objective of this is to enable switching between snap packages and deb packages in a more composable way.

@NucciTheBoss
Copy link
Member

I think we're on the right path here with enabling support for Slurm in both snap and deb form, but I'm wondering where we can "duplicate" things between package formats such that we don't need to have as much abstraction happening in the background to manipulate things like the munge key file and Slurm configuration in the background.

Re. the Slurm configuration, we can just manipulate that with slurmutils. For the munge key, we need to enable support for changing the key file location in mungectl.

@jedel1043
Copy link
Contributor Author

@NucciTheBoss Yeah, using mungectl and slurmutils on both snap and deb mode would be nice. In the meantime, I think we can just push the current abstractions and see if we can unify things later.

@jedel1043
Copy link
Contributor Author

Right, this won't pass CI until we fix charmed-hpc/slurm-snap#47

@jedel1043 jedel1043 marked this pull request as ready for review August 21, 2024 00:06
@NucciTheBoss
Copy link
Member

Right, this won't pass CI until we fix charmed-hpc/slurm-snap#47

Yeah... I think it's time for slurmhelpers to go 😞

@NucciTheBoss NucciTheBoss self-requested a review August 21, 2024 14:25
Copy link
Member

@NucciTheBoss NucciTheBoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great steps in the right direction! I left a couple comments that I would like to know your thoughts on.

Big thing I'm wondering is how we can potentially simplify the creation of the SlurmManagerBase based on which package format we are using.

lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
@jedel1043 jedel1043 requested a review from NucciTheBoss August 21, 2024 18:41
Copy link
Member

@NucciTheBoss NucciTheBoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good! Just a couple comments, but I like how clean this implementation is looking!

lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
@NucciTheBoss
Copy link
Member

@jedel1043 is this PR ready for another review, or are you still working on implementation the configuration management? We could follow on with that in a separate PR since we still need to implement the deb manager, and that PR will give us a better idea of how that implementation needs to shape up.

@jedel1043
Copy link
Contributor Author

@NucciTheBoss We need to integrate slurmutils to make CI pass 😅
Currently working on the update operation for slurmutils.

@NucciTheBoss
Copy link
Member

@NucciTheBoss We need to integrate slurmutils to make CI pass 😅 Currently working on the update operation for slurmutils.

Bout ready to go YOLO and nuke slurmhelpers from the Slurm snap

@jedel1043 jedel1043 marked this pull request as draft September 4, 2024 00:05
@jedel1043
Copy link
Contributor Author

Wow... CI passed...

@jedel1043 jedel1043 force-pushed the slurm-deb-enablement branch 2 times, most recently from 7efe2b0 to f1234df Compare September 6, 2024 22:26
@NucciTheBoss NucciTheBoss self-requested a review September 9, 2024 04:08
Copy link
Member

@NucciTheBoss NucciTheBoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite happy with how this has shaped up. Just a couple suggested changes and I'm ready to merge this so we can get focused on Terraform x AMD GPU stuff 🤩

  1. I don't think we're using the format_key function anymore since we're editing the config files directly.
  2. I think we can worry about linting the passed environment variables later when we have a better idea of where inputs are coming from within the Slurm charms. I think right now we just set/get/unset with no frills about it.
  3. Should we define the Slurm service managers in their own charms respectively? I'm thinking about that discussion we had a while ago where we didn't want to put the managers into this lib because then we'd need to update the lib everywhere everytime we made a change to the manager.

dev-requirements.txt Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
lib/charms/hpc_libs/v0/slurm_ops.py Outdated Show resolved Hide resolved
@NucciTheBoss
Copy link
Member

This is looking great @jedel1043 🎉

I just replied to your comment about the kebab casing for config-server, and I'm good with keeping the individual service managers.

This should enable switching between snaps and debs in any charm using the library.

It also exposes the slurmutils config models instead of having wrapper methods around it, which overall moves the burden of config APIs to slurmutils.
@jedel1043 jedel1043 force-pushed the slurm-deb-enablement branch from 3e1e393 to d606610 Compare September 9, 2024 22:50
@jedel1043 jedel1043 marked this pull request as ready for review September 9, 2024 23:34
Copy link
Member

@NucciTheBoss NucciTheBoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work my man 🎉

I'll open a follow-on PR to do the vbump 😎

@NucciTheBoss NucciTheBoss merged commit 44b96dc into charmed-hpc:main Sep 10, 2024
4 checks passed
@NucciTheBoss
Copy link
Member

On second thought, it would probably be better to implement the deb manager before bumping version. That way the lib is "complete" before building it into the Slurm charms.

@jedel1043 jedel1043 deleted the slurm-deb-enablement branch September 10, 2024 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants